Dynamic Deployment of a MapReduce Architecture in the Cloud
نویسندگان
چکیده
Recently cloud-based MapReduce services have appeared to process large data sets in the Cloud, significantly reducing users’ infrastructure requirements. Almost all these services are Cloud vendor-specific and thus internally designed within their own cloud infrastructure. This leads to two important limitations. Cloud vendors do not provide any clue about how they manage the MapReduce architecture internally hampering its evaluation and also users are not able to either build their own private cloud infrastructure based offering or to use different public cloud infrastructures for this purpose. Thus, this paper describes an architecture which enables the dynamic deployment of a MapReduce architecture in virtual infrastructures provided by either public or private cloud providers. This architecture has been implemented and validated as a proof of concept and released to the community.
منابع مشابه
Cloud Computing Technology Algorithms Capabilities in Managing and Processing Big Data in Business Organizations: MapReduce, Hadoop, Parallel Programming
The objective of this study is to verify the importance of the capabilities of cloud computing services in managing and analyzing big data in business organizations because the rapid development in the use of information technology in general and network technology in particular, has led to the trend of many organizations to make their applications available for use via electronic platforms hos...
متن کاملA Model-driven Approach for Price/Performance Tradeoffs in Cloud-based MapReduce Application Deployment
This paper describes preliminary work in developing a modeldriven approach to conducting price/performance tradeo s for Cloudbased MapReduce application deployment. The need for this work stems from the signi cant variability in both the MapReduce application characteristics and price/performance characteristics of the underlying cloud platform. Our approach involves a model-based machine learn...
متن کاملA Mobile and Fog-based Computing Method to Execute Smart Device Applications in a Secure Environment
With the rapid growth of smart device and Internet of things applications, the volume of communication and data in networks have increased. Due to the network lag and massive demands, centralized and traditional cloud computing architecture are not accountable to the high users' demands and not proper for execution of delay-sensitive and real time applications. To resolve these challenges, we p...
متن کاملAdaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملAn Elastic Middleware Platform for Concurrent and Distributed Cloud and MapReduce Simulations
Cloud Computing researches involve a tremendous amount of entities such as users, applications, and virtual machines. Due to the limited access and often variable availability of such resources, researchers have their prototypes tested against the simulation environments, opposed to the real cloud environments. Existing cloud simulation environments such as CloudSim and EmuSim are executed sequ...
متن کامل